Compact Margin Machine

3 How to utilize data more sufficiently is a crucial consideration in machine learning. Semi-supervised learning uses both unlabeled data and labeled data for this reason. However, Semi-Supervised Support Vector Machine (S3VM) focuses on maximizing margin only, and it abandons the instances which are not support vectors. This fact motivates us to modify maximum margin criterion to incorporate the global information contained in both support vectors and common instances. In this paper, we propose a new method, whose special variant is a semisupervised extension of Relative Margin Machine, to utilize data more sufficiently based on S3VM and LDA. We employ Concave-Convex Procedure to solve the optimization that makes it practical for large-scale datasets, and then give an error bound to guarantee the classifier’s performance theoretically. The experimental results on several real-world datasets demonstrate the effectiveness of our method.